Sensor localization from external source data

ABSTRACT

A system includes a computer including a processor and a memory storing instructions executable by the processor to identify a location and an orientation of a vehicle on a map. The instructions include instructions to determine a location of an infrastructure sensor on the map based on the location and the orientation of the vehicle, data from a vehicle sensor, and data from the infrastructure sensor.

BACKGROUND

A vehicle may operate in an autonomous mode, a semiautonomous mode, or anonautonomous mode. In the autonomous mode, each of a propulsion system,a braking system, and a steering system are controlled by a vehiclecomputer; in a semiautonomous mode the vehicle computer controls one ortwo of the propulsion system, the braking system, and the steeringsystem; in a nonautonomous mode, a human operator controls thepropulsion system, the braking system, and the steering system. Thevehicle may operate in the autonomous mode and the semiautonomous modebased at least in part on data from an infrastructure sensor.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of components of a system for determining alocation of an infrastructure sensor on a map.

FIG. 2 is an illustration of a map including the infrastructure sensorand a vehicle.

FIG. 3 is an illustration of data collected by a sensor of the vehicleand specifying a point cloud.

FIG. 4 is an illustration of data collected by the infrastructure sensorand specifying a point cloud.

FIG. 5 is a flow chart illustrating a process for controlling thesystem.

DETAILED DESCRIPTION

A system includes a computer including a processor and a memory storinginstructions executable by the processor to identify a location and anorientation of a vehicle on a map. The instructions include instructionsto determine a location of an infrastructure sensor on the map based onthe location and the orientation of the vehicle, data from a vehiclesensor, and data from the infrastructure sensor.

The instructions may further include instructions to identify thelocation and the orientation of the vehicle based on a location of anobject on the map and a location of the object relative to the vehicle.

The instructions may further include instructions to identify a firstlocation of an object on the map and based on the data from the vehiclesensor, to identify a second location of the object relative to theinfrastructure sensor and based on the data from the infrastructuresensor, and to determine the location of the infrastructure sensor basedon the first and second locations of the object.

The instructions may further include instructions to identify a firstplane based on the data from the vehicle sensor, to identify a secondplane based on the data from the infrastructure sensor, and to determinethe location of the infrastructure sensor based on the first plane andthe second plane.

The instructions may further include instructions to identify a firstvector based on the data from the vehicle sensor, to identify a secondvector based on the data from the data from the infrastructure sensor,and to determine the location of the infrastructure sensor based on thefirst vector and the second vector.

The data from the vehicle sensor and the data from the infrastructuresensor may include point-cloud data.

The instructions may further include instructions to identify a secondlocation and a second orientation of the vehicle relative to the map,collect second data from the vehicle sensor while the vehicle is at thesecond location and in the second orientation, and to determine thelocation of the infrastructure sensor on the map based on the locationand the orientation of the vehicle, the data from the vehicle sensor,the second location and the second orientation of the vehicle, thesecond data from the vehicle sensor, and the data from theinfrastructure sensor.

The instructions may further include instructions to determine anorientation of the infrastructure sensor on the map based on thelocation and the orientation of the vehicle, the data from the vehiclesensor, and the data from the infrastructure sensor.

The computer may be remote from the vehicle and the infrastructuresensor, and the instructions may further include instructions to storethe location of the infrastructure sensor on the map in the memory ofthe computer.

The instructions may further include instructions to store the locationof the infrastructure sensor on the map in a memory of theinfrastructure sensor.

The instructions may further include instructions to navigate a secondvehicle based on the location of the infrastructure sensor.

A method includes identifying a location and an orientation of a vehicleon a map. The method includes determining a location of aninfrastructure sensor on the map based on the location and theorientation of the vehicle, data from a vehicle sensor, and data fromthe infrastructure sensor.

The method may further include identifying the location and theorientation of the vehicle based on a location of an object on the mapand a location of the object relative to the vehicle.

The method may further include identifying a first location of an objecton the map and based on the data from the vehicle sensor, identifying asecond location of the object relative to the infrastructure sensor andbased on the data from the infrastructure sensor, and determining thelocation of the infrastructure sensor based on the first and secondlocations of the object.

The method may further include identifying a first plane based on thedata from the vehicle sensor, identify a second plane based on the datafrom the infrastructure sensor, and determining the location of theinfrastructure sensor based on the first plane and the second plane.

The method may further include identifying a first vector based on thedata from the vehicle sensor, identifying a second vector based on thedata from the data from the infrastructure sensor, and determining thelocation of the infrastructure sensor based on the first vector and thesecond vector.

The data from the vehicle sensor and the data from the infrastructuresensor may include point-cloud data.

The method may further include identifying a second location and asecond orientation of the vehicle on the map, collecting second datafrom the vehicle sensor while the vehicle is at the second location andin the second orientation, and determining the location of theinfrastructure sensor on the map based on the location and theorientation of the vehicle, the data from the vehicle sensor, the secondlocation and the second orientation of the vehicle, the second data fromthe vehicle sensor, and the data from the infrastructure sensor.

The method may further include determining an orientation of theinfrastructure sensor relative to the map based on the location and theorientation of the vehicle, the data from the vehicle sensor, and thedata from the infrastructure sensor.

The method may further include storing the location of theinfrastructure sensor on the map in a memory of the infrastructuresensor.

The method may further include navigating a second vehicle based on thelocation of the infrastructure sensor.

A computer may have a processor and a memory storing instructionsexecutable by the processor to perform the method.

The computer may be remote from the vehicle and the infrastructuresensor, and the instructions may further include instructions to storethe location of the infrastructure sensor on the map in the memory ofthe computer.

A computer readable medium may store instructions executable by aprocessor to perform the method.

Referring to FIGS. 1 and 2 a system 20 provides for determining alocation of an infrastructure sensor 22 on a map 26. The system 20includes a computer 34, 36 having a processor and a memory storinginstructions executable by the processor to identify a location and anorientation of a vehicle 24 on the map 26. The instructions includeinstructions to determine a location of the infrastructure sensor 22 onthe map 26 based on the location and the orientation of the vehicle 24,data from a sensor 28 of the vehicle 24, and data from theinfrastructure sensor 22.

The system 20 enable the location of the infrastructure sensor 22 on themap 26 to be determined without requiring that the infrastructure sensor22 include specific hardware, e.g., a GPS system. Determining thelocation of the infrastructure sensor 22 on the map 26 provides for datafrom the infrastructure sensor 22 to be used to identify a location ofone or more objects, such as vehicles, on the map 26. Determining thelocation of the infrastructure sensor 22 on the map 26 also provides fordata from the infrastructure sensor 22 to be fused with other data, suchas data from other infrastructure sensors and/or vehicles, e.g., foroperating a vehicle in an autonomous mode and/or a semiautonomous mode.

The map 26 is illustrated in FIG. 2 as a conventional graphical map butincludes a set of data that can be stored in digital format, which canbe referred to as map data, that specifies physical features and objects30, e.g., roads and/or road segments (including highways, roads, citystreets, etc.), lanes, bridges, buildings, infrastructure, signs,traffic-lights, etc., at respective locations in a defined area. The map26 may include data specifying shapes or boundaries of the objects 30.The shapes may be three-dimensional (3-D), e.g., the data may specifyheights, widths, and depths of surfaces of the objects 30. The map 26may include data specifying orientations (i.e., relative positions ordirections in a 3-D coordinate system) of the objects 30 on the map 26.The data may specify coordinates of the physical features and objects30, e.g., latitude and longitude geocoordinates, X-Y-Z coordinatesrelative to specified X-Y-Z axes having a specified origin, etc. Somecoordinates, e.g., X-Y coordinates may be relative to a positioningsystem (such as GPS), relative to a certain physical feature (such as anintersection of roads or other object 30), or relative to any othersuitable datum or data for defining locations on the map 26. The data ofthe map 26 may be based on cartography documents, a geographical survey,a previously stored map (or at least a portion thereof), or otherinformation suitable for specifying locations of physical features andobjects 30 in a defined area. The computer 34, 36 can store map data asis conventionally known, e.g., for use in a vehicle navigation system 32or the like.

A location on the map 26 specifies where a physical feature or object 30is on the map 26, e.g., specified X-Y-Z coordinates, GPS coordinates,etc. An orientation on the map 26 specifies a facing direction of aspecified surface of the physical feature or object 30 on the map 26,e.g., a specified compass heading direction, angle relative to the X,Y,and Z axes, etc. Data specifying a location and an orientation of aphysical feature or object 30 may be stored independent of the map 26.For example, a location and an orientation of the infrastructure sensor22 on the map 26 may be stored in memory of the infrastructure sensor 22and the map 26 may be stored in memory of a vehicle computer 34 and/or aserver computer 36.

The system 20 can use a network 38 to provide communication amongcomponents of the system 20. The network 38 (sometimes referred to asthe wide area network 38 because it can include communications betweendevices that are geographically remote from one another, i.e., not in asame building, vehicle 24, etc.) represents one or more mechanisms bywhich remote devices, e.g., the server computer 36, the vehicle 24, theinfrastructure sensor 22, etc., may communicate with each other.Accordingly, the network 38 may be one or more wired or wirelesscommunication mechanisms, including any desired combination of wired(e.g., cable and fiber) and/or wireless (e.g., cellular, wireless,satellite, microwave, and radio frequency) communication mechanisms andany desired network topology (or topologies when multiple communicationmechanisms are utilized).

The vehicle 24 may be any type of passenger or commercial vehicle suchas a car, a truck, a sport utility vehicle, a crossover vehicle, a van,a minivan, a taxi, a bus, etc. Although illustrated as a passengervehicle, the vehicle 24 may be unmanned, e.g., a land-based or aerialdrone.

The vehicle 24 may operate in an autonomous mode, a semiautonomous mode,or a nonautonomous mode. For purposes of this disclosure, an autonomousmode is defined as one in which each of a propulsion system 40, abraking system 42, and a steering system 44 are controlled by thevehicle computer 34; in a semiautonomous mode the vehicle computer 34controls one or two of the propulsion system 40, the braking system 42,and the steering system 44; in a nonautonomous mode, a human operatorcontrols the propulsion system 40, the braking system 42, and thesteering system 44.

The vehicle 24 includes sensors 28. The sensors 28 may detect internalstates of the vehicle 24, for example, wheel speed, wheel orientation,and engine and transmission variables. The sensors 28 may detect theposition or orientation of the vehicle 24, for example, globalpositioning system (GPS) sensors; accelerometers such as piezo-electricor microelectromechanical systems (MEMS) sensors; gyroscopes such asrate, ring laser, or fiber-optic gyroscopes; inertial measurements units(IMU); and magnetometers. The sensors 28 may detect the external world,for example, radar sensors, scanning laser range finders, lightdetection and ranging (LIDAR) devices, and image processing sensors suchas cameras. Lidar sensors measure distances to detected objects 30 byilluminating the object 30 with pulsed laser light and measuring returntimes of reflected pulses. Differences in return times and wavelengthsof reflected pulses can then be used to generate data specifying a pointcloud. The sensors 28 may include communications devices, for example,vehicle-to-infrastructure (V2I) or vehicle-to-vehicle (V2V) devices.

A sensor 28 defines a field of view 29 (illustrated in FIGS. 2 and 3 ).The field of view 29 of each sensor 28 is a volume relative to, anddetectable, by such sensor 28. The volume may be defined by azimuth andaltitude angle ranges, as well as by a depth, or detection distance.

The propulsion system 40 translates energy into motion of the vehicle24, e.g., in response to an instruction from the computer 34 and/or inresponse to an operator input, such as to an accelerator pedal. Forexample, the propulsion system 40 may include a conventional powertrainhaving an internal-combustion engine coupled to a transmission thattransfers rotational motion to wheels; an electric powertrain havingbatteries, an electric motor, and a transmission that transfersrotational motion to the wheels; a hybrid powertrain having elements ofthe conventional powertrain and the electric powertrain; or any othertype of structure for providing motion to the vehicle 24. In the case ofan aerial vehicle, the propulsion system 40 may include one more motorsoperatively coupled to one or more propellers. The motors provide torquethat rotates the propellers, e.g., to generate thrust and control apitch, roll, and/or yaw of an aerial drone. The propulsion system 40 canbe controlled by, and may report data via, an electronic control unit(ECU) or the like in communication with the computer 34.

The braking system 42 resists motion of the vehicle 24 to thereby slowand/or stop the vehicle 24, e.g., in response to an instruction from thevehicle computer 34 and/or in response to an operator input, such as toa brake pedal. The braking system 42 may include friction brakes such asdisc brakes, drum brakes, band brakes, and so on; regenerative brakes;any other suitable type of brakes; or a combination. The braking system42 may be controlled by, and may report data via, an electronic controlunit (ECU) or the like in communication with the vehicle computer 34.

The steering system 44 and controls the turning of wheels of the vehicle24. The steering system 44 is in communication with and receives inputfrom a steering wheel and the vehicle computer 34. The steering system44 may be a rack-and-pinion system with electric power-assistedsteering, a steer-by-wire system, as are both known in the art, or anyother suitable system.

The vehicle 24 may include the navigation system 32. The navigationsystem 32 is implemented via circuits, chips, or other electroniccomponents that can determine a present location of the vehicle 24. Thenavigation system 32 may be implemented via satellite-based system suchas the Global Positioning System (GPS). The navigation system 32 maytriangulate the location of the vehicle 24 based on signals receivedfrom various satellites in the Earth's orbit. The navigation system 32is programmed to output signals representing the present location of thevehicle 24. The navigation system 32 may use data from sensors 28 of thevehicle 24, e.g., wheel speed sensors and a magnetometer, to furtherdetermine the location of the vehicle 24. In some instances, thenavigation system 32 is programmed to determine a route from the presentlocation to a future location, including developing alternative routes,e.g., when a road is closed or congested. The navigation system 32 mayaccess a map 26 stored in the memory of the vehicle computer 34(discussed below) and develop the route according to the map 26.

The vehicle 24 may include a communication network 46 includinghardware, such as a communication bus, for facilitating communicationamong components of the vehicle 24, such as the computer 34, thepropulsion system 40, the braking system 42, the steering system 44, thesensors 28, and the navigation system 32. The communication network 46may facilitate wired or wireless communication among the components inaccordance with a number of communication protocols such as controllerarea network (CAN), Ethernet, WiFi, Local Interconnect Network (LIN),and/or other wired or wireless mechanisms.

The vehicle computer 34, implemented via circuits, chips, and/or otherelectronic components, is included in the system 20 for carrying outvarious operations, including as described herein. The vehicle computer34 is a computing device that generally includes a processor and amemory, the memory including one or more forms of computer-readablemedia and storing instructions executable by the processor forperforming various operations, including as disclosed herein. The memoryof the vehicle computer 34 further generally stores remote data receivedvia various communications mechanisms; e.g., the vehicle computer 34 isgenerally configured for communications on the communication network 46or the like, and/or for using other wired or wireless protocols, e.g.,Bluetooth, etc. The vehicle computer 34 may also have a connection to anonboard diagnostics connector (OBD-II). Via the communication network 46and/or other wired or wireless mechanisms, the vehicle computer 34 maytransmit and receive messages to and from various devices in the vehicle24, e.g., steering system 44, the braking system 42, the propulsionsystem 40, the navigation system 32, the sensors 28, etc. Although onevehicle computer 34 is shown in FIG. 1 for ease of illustration, it isto be understood that the vehicle computer 34 could include, and variousoperations described herein could be carried out by, one or morecomputing devices.

The vehicle computer 34 is programmed to, i.e., the memory of thevehicle computer 34 stores instructions executable by the processor ofthe vehicle computer 34 to, identify a location and an orientation ofthe vehicle 24 on the map 26. The vehicle computer 34 may identify thelocation and orientation of the vehicle 24 based on data from thenavigation system 32 and the sensors 28. The vehicle computer 34 mayidentify the location and the orientation of the vehicle 24 based on alocation and orientation of an object 30, e.g., a building,traffic-signal, etc., on the map 26 and a location of the object 30detected by the sensors 28 and relative to the vehicle 24. The vehiclecomputer 34 may identify the location and orientation of the vehicle 24with other techniques, such as those using conventional geolocation(e.g., GPS), real-time kinematics (RTK), visual and/or LIDAR odometry,and inertial measurement unit (IMU) data.

For example, the vehicle computer 34 may identify an object 30 detectedby the sensors 28 as being an object 30 on the map 26, e.g., specifiedby the map data. To identify the object 30, the vehicle computer 34 maycompare one or more parameters, e.g., length, width, curvature, etc.,specified by the data generated with a LIDAR sensor with one or moreparameters specified by the data of the object 30 on the map 26. Thevehicle computer 34 may identify object 30 in the data from the LIDARsensor as the object 30 on the map 26 when the parameters specified insuch data match, or are within a threshold amount of matching, e.g.,95%. Additionally or alternatively, the vehicle computer 34 may identifythe object 30 based on image recognition analysis of data specifying oneor more images captured by a camera. The objects 30 may be recognized inthe image data using conventional techniques and methods, e.g.,parameters of such objects 30 may be compared to the parametersspecified by the data of the object 30 on the map 26.

The vehicle computer 34 may determine a location and orientation of thedetected object 30 relative to the vehicle 24, e.g., a distance from thevehicle 24 to the object 30, and a direction of the object 30 from thevehicle 24 (such as angle to the right or left of vehicle forward). Thelocation and orientation of the detected object 30 relative to thevehicle 24 may be determined based on data from sensors 28. For example,time of flight data from LIDAR sensor, stereo image analysis of imagesfrom a pair of cameras, or with other conventional techniques. Thevehicle computer 34 may combine the identified location and orientationof the detected object 30 relative to the vehicle 24 with the locationand orientation of the object 30 on the map 26 to identify a locationand an orientation of the vehicle 24 on the map 26, e.g., usingconventional techniques.

The vehicle computer 34 may be programmed to identify planes 48 in apoint cloud 31 (illustrated in FIGS. 3 and 4 ) specified by data from aLIDAR sensor. The vehicle computer 34 may identify a plane 48 byselecting three points 33 of the point cloud 31 that are close together,typically points 33 where there are no other points 33 between any pairof selected points 33. The vehicle computer 34 may define the plane 48based on the selected three points 33, i.e., such that all points 33 onthe plane 48. The computer 34 may then identify additional points 33 ofthe point cloud 31 that are within a threshold distance e.g., 10centimeters, of at least one selected point 33 and are within athreshold distance, e.g., 10 centimeters, of the plane 48 along an axisnormal to the plane 48. Such identified additional points 33 may also beselected in addition to the previously selected three points 33.

The vehicle computer 34 may re-define the plane 48 based on all selectedpoints 33, e.g., such that the plane 48 is best fit to the points 33.For example, the vehicle computer 34 may best fit the plane 48 usingconventional techniques. The vehicle computer 34 may then identifyadditional points 33 of the point cloud 31 that are within the thresholddistance of at least one selected point and are within the thresholddistance of the plane 48. The vehicle computer 34 may select such points33 and again redefine the plane 48, e.g., repeating iteratively until nomore points 33 can be identified within the threshold distance of atleast one selected point 33 and within the threshold distance of theplane 48. The vehicle computer 34 may identify the planes 48 with thedistribution of points 33 in the point cloud 31 by identifying a subsetof points 33 with the greatest number of points 33 that can berepresented by one of the planes 48 with small error. The vehiclecomputer 34 estimates an equation specifying the plane 48 and removespoints 33 from the point cloud 31 hierarchically. The equationspecifying the plane 48 may be identified with the Eigen value/vectordecomposition of the covariance matrix of the subset of points 33 fromthe point cloud 31. The computer 34 may identify the planes 48 withother techniques, such as those conventionally known.

The planes 48 may be bounded by one or more intersections 50 with one ormore other planes 48 and/or at edges 52 defined by the selected points33. The vehicle computer 34 may identify one or more corners 54 wherethe intersections 50 and/or edges 52 meet.

The vehicle computer 34 may identify the position of the plane 48, e.g.,the position of one or more corners 54 of the plane 48, of a center ofthe plane 48, relative to the vehicle 24. For example, the vehiclecomputer 34 may identify that one of the corners 54 of the plane 48 is20 meters from the vehicle 24, and at an azimuth angle of 20 degrees tothe right of vehicle forward and an altitude angle of 5 degrees upward.The vehicle computer 34 may additionally determine the positions of thecorners 54 relative to each other.

The vehicle computer 34 may identify an orientation of the plane 48,e.g., relative to a facing direction of the vehicle 24. For example, thevehicle computer 34 may identify the plane 48 is elongated and extending20 degrees to the left and 0 degrees upward relative to a forwarddirection of the vehicle 24. The vehicle computer 34 may determine theelongation and extension of the plane 48 based on the selected points33, e.g., along an edge 52 or intersection 50, along a longest lineconnecting adjacent corners 54 (e.g., adjacent corners 54 of the plane48 that are spaced furthest from each other relative to spacing of othercorners 54 from each other).

The vehicle computer 34 may be programmed to identify vectors 56 in thepoint cloud 31. The vehicle computer 34 may identify a vector 56 basedon a plane 48. For example, a root, or starting location, of the vector56 may be at a center of the plane 48. The center of the plane 48 may bea geometrically weighted center of the selected points of the pointcloud data used to define the plane 48. The vector 56 may extendnormally (perpendicularly) with respect to the plane 48. The vector 56may have a length based on a size, e.g., in meters squared, of the plane48. For example, the greater the size, the longer the vector 56. Thelength of the vector 56 may be linearly scaled to the size of the plane48.

The vehicle computer 34 may identify the position of the vector 56,e.g., the position of the root of the vector 56, relative to the vehicle24. For example, the computer may identify the root of the vector 56 is15 meters from the vehicle 24, and at an azimuth angle of 50 degrees tothe right of vehicle forward and an altitude angle of 15 degrees upward.The vehicle computer 34 may identify an orientation of the vector 56,e.g., relative to a facing direction (i.e., a forward direction along alongitudinal axis) of the vehicle 24. For example, the vehicle computer34 may identify the vector 56 is extending 15 degrees to the left and 0degrees upward relative to a forward direction of the vehicle 24.

The vehicle computer 34 may identify a location and orientation ofobject 30, the plane 48 and/or vector 56 on the map 26. For example, thevehicle computer 34 may use the position and orientation of the vehicle24 on the map 26 as a starting datum and may determine the position andorientation of the object 30, the plane 48 and/or the vector 56 on themap 26 based on the position and orientation of the plane 48 and/orvector 56 relative to the vehicle 24. In other words, the relativeposition and orientation of the plane 48 and/or vector 56 relative tothe vehicle 24 may be combined with the location and orientation of thevehicle 24 on the map 26.

The vehicle 24 may identify multiple locations and orientations of theobject 30, the plane 48 and/or vector 56 on the map 26, e.g., while thevehicle 24 is at different locations on the map 26. The vehicle computer34 may identify an average location and orientation of the multiplelocations and orientations of the object 30, plane 48, and/or vector 56as the location and the orientation of the object 30, plane 48 and/orthe vector 56 on the map 26. The average location of the object 30,plane 48, and/or vector 56 is a location on the map that expresses thetypical, e.g., mean, location of the multiple identified locations ofthe object 30, plane 48, and/or vector 56 on the map. The averageorientation of the object 30, plane 48, and/or vector 56 on the map isan orientation that expresses the typical orientation of the multipleidentified orientations of the object 30, plane 48, and/or vector 56 onthe map. The average location may be standard vector averaging, whileaverage for orientation may be an average in the Rimmenian manifoldwhich only includes numbers in the space of possible orientations (e.g.,−180 to 180 degrees or 0-360, alternatively). The vehicle computer 34may identify the average location of the multiple locations byidentifying averages of the individual conditionate locations, e.g., bysumming all the x-coordinates of the multiple locations of the object 30on the map and dividing by the number of the multiple locations, and soon for the y-coordinates and z-coordinates. The vehicle computer 34 mayidentify the average orientation of the multiple orientations byidentifying the average facing direction of a common (or same) specifiedsurface of the physical feature or object 30 on the map 26, e.g., bysumming all the compass heading directions (or angle relative to theX,Y, and Z axes) of the multiple orientations on the map and dividing bythe number of the multiple orientations. The vehicle computer 24 may useother conventional techniques to identify the average location andorientation of the multiple locations and orientations of the object 30,plane 48, and/or vector 56 on the map.

The vehicle computer 34 may be programmed to transmit data, e.g., to theserver computer 36 and/or the infrastructure sensor 22, e.g., via thewide area network 38. The transmitted data may be from the sensors 28and may specify the external world, e.g., image data from a camera, orpoint cloud data from a LIDAR sensor. The transmitted data may specifyone or more objects 30, planes 48, and/or vectors 56 defined by thedata. The transmitted data may specify locations and orientations of theobjects 30, planes 48, and/or vectors 56. The locations and orientationsof the objects 30, planes 48, and/or vectors 56 may be on the map 26and/or relative to the vehicle 24. The transmitted data may specify thelocation and orientation of the vehicle 24 on the map 26. The vehiclecomputer 34 may transmit data in response to receiving a request forsuch data, e.g., from the server computer 36 and/or the infrastructuresensor 22.

The vehicle computer 34 may be programmed to navigate the vehicle 24.For example, the vehicle computer 34 may transmit commands to thesteering system 44, the propulsion system 40, and/or the braking system42. The commands may specifically actuation of the systems 40, 42, 44 tonavigate the vehicle 24 along a route, i.e. a specified path from pointA to point B, while avoiding obstacles. The vehicle computer 34 maydetermine the commands based on data from the navigation system 32 andthe sensors 28, e.g., specifying a location of the vehicle 24, a headingof the vehicle 24, a route, one or more detected objects 30 (such asother vehicles, pedestrians, buildings, lane markers, etc.). Thecomputer may navigate the vehicle 24 based on the data from theinfrastructure sensor 22. For example, LIDAR or image data from theinfrastructure sensor 22 may be fused with data from the sensors 28 ofthe vehicle 24. As another example, data from the infrastructurestructure sensor may specify locations of detected objects 30, e.g.,locations of other vehicles on the map 26. The vehicle computer 34 maynavigate the vehicle 24 based on data from the sensors 28, thenavigation system 32, and/or the infrastructure sensor 22 withconventional techniques.

The infrastructure sensor 22 is a sensor assembly in a fixedgeographical location, e.g., fixed to a building, bridge, street post,etc. The infrastructure sensor 22 detects the external world, forexample, the infrastructure sensor 22 may include radar sensors,scanning laser range finders, light detection and ranging (LIDAR)devices, and image processing sensors such as cameras. Theinfrastructure sensor 22 defines a field of view 27 (illustrated inFIGS. 2 and 4 ). The infrastructure sensor 22 may include communicationsdevices, for example, configured to provide communication with thevehicle computer 34 and the server computer 36 via the wide area network38. The infrastructure sensor 22 may include a computer, implemented viacircuits, chips, and/or other electronic components. The computer is acomputing device that generally includes a processor and a memory, thememory including one or more forms of computer-readable media andstoring instructions executable by the processor for performing variousoperations, including as disclosed herein.

The infrastructure sensor 22 may be programmed to, i.e., the computer ofthe infrastructure sensor 22 may be programmed to, identify objects 30,planes 48, and/or vectors 56, including their respective locations andorientations relative to the infrastructure sensor 22, e.g., asdescribed for the vehicle computer 34.

The infrastructure sensor 22 may be programmed to identify locations andorientations of objects 30, planes 48, and/or vectors 56 on the map,e.g., as described for the vehicle computer 34 and after the locationand orientation of the infrastructure sensor 22 on the map is stored inmemory of the infrastructure sensor 22.

The infrastructure sensor 22 may be programmed to transmit data, e.g.,to the vehicle 24, the server computer 36, etc., via the wide areanetwork 38. The data transmitted by the infrastructure sensor 22 mayspecify the external world detected by the sensor, e.g., image data froma camera, data specifying a point cloud 31 generated from a LIDARsensor, etc. The transmitted data may specify one or more objects 30,planes 48, and/or vectors 56 defined by the data. The transmitted datamay specify locations and orientations of the objects 30, planes 48,and/or vectors 56. The locations and orientations of the objects 30,planes 48, and/or vectors 56 may be relative to the infrastructuresensor 22. The locations and orientations of the objects 30, planes 48,and/or vectors 56 may be relative to a location and orientation of theinfrastructure sensor 22 on the map 26.

The infrastructure sensor 22 may be programmed to may be programmed tonavigate a second vehicle (not shown) based on the location andorientation of the infrastructure sensor 22 on the map 26. For example,the infrastructure sensor 22 may detect objects 30 (and identify theirlocations and orientations on the map 26), such as other vehicles, thatare not detectable by sensors 28 of the second vehicle. Theinfrastructure sensor 22 may transmit data specifying the locations andorientations of detected objects 30 on the map 26 to the second vehicle,e.g., so the second vehicle may navigate to avoid impact with suchobjects 30. As another example, the infrastructure sensor 22 maytransmit a command, e.g., instructing the second vehicle to stop at anintersection of roads on the map 26 when data collected by theinfrastructure sensor 22 specifies cross traffic at the intersection.

The server computer 36 is remote from the vehicle 24 and theinfrastructure sensor 22. The server computer 36 may be one or morecomputer servers, each including at least one processor and at least onememory, the memory storing instructions executable by the processor,including instructions for carrying out various steps and processesdescribed herein. The server computer 36 may include or becommunicatively coupled to a data store for storing collected data.

The server computer 36 may be programmed to identify a location and anorientation of the vehicle 24 on the map 26. The server computer 36 mayidentify the location and the orientation of the vehicle 24 by receivingdata from the vehicle 24 specifying the location and the orientation ofthe vehicle 24 on the map 26. The server computer 36 may identify thelocation and the orientation of the vehicle 24 based on data received bythe server computer 36 and generated by sensors 28 of the vehicle 24,data generated sensors 28 of a second vehicle and/or an infrastructuresensor 22 having an identified location on the map 26, etc., e.g., asdescribed for the vehicle computer 34 and/or with conventionaltechniques.

The server computer 36 may be programmed to identify objects 30, planes48 and vectors 56 based on data received from the vehicle 24 and/orinfrastructure sensor 22, including respective locations and orientationrelative to the vehicle 24 and/or the infrastructure sensor 22, e.g., asdescribed for the vehicle computer 34.

The server computer 36 may identify a first plane 48 identified in afirst point cloud 31 generated by the infrastructure sensor 22 and asecond plane 48 identified in a second point cloud 31 generated by thesensor 28 of the vehicle 24 as being a same plane 48. In other words, inthis context same planes 48 are planes 48 defined by different pointclouds 31 that specify a common surface. For example, the points 33 ofthe respective point clouds 31 may specify a common wall or other object30 surface detected by both the infrastructure sensor 22 and sensors 28of the vehicle 24.

The server computer 36 may identify the planes 48 identified in thedifferent point clouds 31 as being the same based on relative positionsof corners 54, intersections, 50 and/or edges 52 of each plane 48 beingsubstantially the same. The server computer 36 may identify the planes48 identified in the point clouds 31 from the infrastructure sensor 22and the sensor 28 of the vehicle 24 as being the same based on similarrelationships between the planes 48 and other planes 48 identified inthe respective point clouds 31. For example, the server computer 36 mayidentify an angle defined between a first plane 48 and a second plane 48in the point cloud 31 generated by the infrastructure sensor 22. Theserver computer 36 may identify an angle defined between a first plane48 and a second plane 48 in the point cloud 31 generated by the sensor28 of the vehicle 24. The server computer 36 may identify the firstplanes 48 as being same planes 48 when the angles are substantially thesame.

The server computer 36 may identify a vector 56 identified in a pointcloud 31 generated by the infrastructure sensor 22 and a vector 56identified in a point cloud 31 generated by the sensor 28 of the vehicle24 as being same vectors 56. Same vectors 56 are vectors 56 defined bypoint clouds 31 that specify a common surface specified by the points 33of the point clouds 31, e.g., a common wall detected by both theinfrastructure sensor 22 and sensors 28 of the vehicle 24.

The server computer 36 may identify the vectors 56 identified in thedifferent point clouds 31 as being the same based on a length of thevectors 56 being substantially the same and/or based on similarrelationships between the vectors 56 and other vectors 56 identified inthe respective point clouds 31. For example, the server computer 36 mayidentify a distance between, and a relative orientation of, a firstvector 56 and a second vector 56 in the point cloud 31 generated by theinfrastructure sensor 22. The server computer 36 may also identify adistance between, and a relative orientation of, a first vector 56 and asecond vector 56 in the point cloud 31 generated by the sensor 28 of thevehicle 24. The server computer 36 may identify the first vectors 56 asbeing same vectors 56 when the distances and relative orientations aresubstantially the same.

The server computer 36 is programmed to determine a location and anorientation of the infrastructure sensor 22 on the map 26 based on thelocation and the orientation of the vehicle 24 on the map, the data fromthe sensor 28 of the vehicle 24, and the data from the infrastructuresensor 22. The data from the sensor 28 of the vehicle 24 may specifylocations and orientations of objects 30, planes 48, and/or vectors 56on the map 26 and/or relative to the vehicle 24. The data from theinfrastructure sensor 22 may specify locations and orientations ofobjects 30, planes 48, and/or vectors 56 relative to the infrastructuresensor 22.

The server computer 36 may determine the location and the orientation ofthe infrastructure sensor 22 on the map 26 by combining the location andorientation of the vehicle 24 on the map 26 with the location andorientation of the object 30, the plane 48, and/or the vector 56relative to the vehicle 24, and with the location and orientation of theobject 30, the plane 48, and/or the vector 56 relative to theinfrastructure sensor 22.

The server computer 36 may determine the location and the orientation ofthe infrastructure sensor 22 on the map 26 by first identifying thelocation and orientation the location of the object 30, the plane 48,and/or the vector 56 on the map 26 (or the average location andorientation as described above), and then determining the orientation ofthe infrastructure sensor 22 on the map 26 by combining the location andthe orientation of the object 30, the plane 48, and/or the vector 56 onthe map 26 with the position and orientation of the object 30, the plane48, the and/or vector 56 relative to the infrastructure sensor 22. Theserver computer 36 may determine the location and orientation of theinfrastructure sensor 22 on the map 26 using triangulation or otherconventional techniques.

The server computer 36 may combine the location and the orientation ofthe object 30, the plane 48, and/or the vector 56 on the map 26 with theposition and orientation of the object 30, the plane 48, the and/orvector 56 relative to the infrastructure sensor 22 by aligning therespective the objects 30, the planes 48, and/or the vectors 56 in thedata from the sensor 28 of the vehicle 24 with the respective theobjects 30, the planes 48, and/or the vectors 56 in the data from theinfrastructure sensor 22. The server computer 36 aligns same objects 30,planes 48, and/or vectors 56 in the respective data such that the sameobjects 30, planes 48, and/or vectors 56 overlap, i.e., are on a samelocation and orientation on the map 26. For example, the first andsecond planes 48 a, 48 b of the data from the infrastructure sensor 22may be aligned to overlap the first and second planes 48 a, 48 b of thedata from the sensor 28 of the vehicle 24. As another example, the firstand second vectors 56 a, 56 b of the data from the infrastructure sensor22 may be aligned to overlap the first and second vectors 56 a, 56 b ofthe data from the sensor 28 of the vehicle 24. Aligning objects 30, theplanes 48, and/or the vectors 56 is advantageously more efficient, i.e.,uses less computing resources and/or time, than aligning point clouds31. The respective the objects 30, the planes 48, and/or the vectors 56in the data from the infrastructure sensor 22 may be aligned with theobjects 30, the planes 48, and/or the vectors 56 in the data from thesensor 28 of the vehicle 24 by solving an optimization problem, such asan objective function Wahba's problem, the lazy projections GDalgorithm, or other conventional technique.

The server computer 36 may identify additional locations andorientations of the vehicle 24 relative to the map 26 while the vehicle24 is at different location and orientations. The server computer 36 maycollect additional data from the vehicle sensor 28 while the vehicle 24is at the additional locations and orientations. The server computer 36may determine additional locations and orientations of theinfrastructure sensor 22 on the map 26 based on the additional locationsand orientations of the vehicle 24, the additional data from the vehiclesensor 28, and the data from the infrastructure sensor 22, e.g., asdescribed above. The computer 36 may identify an average location andorientation of the multiple locations and orientations of theinfrastructure sensor 22 on the map 26, e.g., providing increasedaccuracy to the determined location and orientation of theinfrastructure sensor 22 on the map 26. The average location of theinfrastructure sensor 22 is a location on the map that expresses thetypical, e.g., mean, location of the multiple identified locations ofthe infrastructure sensor 22 on the map. The average orientation of theinfrastructure sensor 22 on the map is an orientation that expresses thetypical orientation of the multiple identified orientations of theinfrastructure sensor 22 on the map. The computer 36 may identify theaverage location of the multiple locations of the infrastructure sensor22 by identifying averages of the individual conditionate locations,e.g., by summing all the x-coordinates of the multiple locations on themap and dividing by the number of the multiple locations, and so on forthe y-coordinates and z-coordinates. The computer 36 may identify theaverage orientation of the multiple orientations of the infrastructuresensor 22 by identifying the average facing direction of theinfrastructure sensor 22, e.g., by summing all the compass headingdirections (or angle relative to the X, Y, and Z axes) of the multipleorientations of the infrastructure sensor 22 on the map and dividing bythe number of the multiple orientations. The computer 36 may use otherconventional techniques to identify the average location and orientationof the infrastructure sensor 22 on the map.

The server computer 36 may store the location and orientation of theinfrastructure sensor 22 on the map 26 in memory of the server computer36. The server computer 36 may transmit the location and orientation ofthe infrastructure sensor 22 on the map 26 to the infrastructure sensor22, e.g., to be stored in memory of the infrastructure sensor 22.

The server computer 36 may be programmed to transmit data specifying thelocation and orientation of the infrastructure sensor 22 on the map 26to the vehicle 24 and/or a second vehicle for autonomous and/orsemiautonomous navigation of the vehicle 24 and/or the second vehicle.For example, the server computer 36 may transmit data specifying thelocation and orientation of the infrastructure sensor 22 on the map 26via the wide area network 38 to the vehicle 24 and/or the secondvehicle, and the vehicle 24 and/or the second vehicle may fuse data frominfrastructure sensor 22 with data from sensors 28 of the second vehicleusing the location and orientation of the infrastructure sensor 22 onthe map 26, e.g., using conventional techniques for fusing data frommultiple sensors for autonomously or semi-autonomously operating avehicle.

FIG. 5 is a process flow diagram illustrating an exemplary process 500for operating the system 20. The process 500 begins in a block 505 wherethe server computer 36 transmits a request to the infrastructure sensor22. The request may specify the infrastructure sensor 22 provide rawdata, e.g., image and/or point cloud data. The request may specify theinfrastructure sensor 22 provide data specifying locations andorientations of objects 30, planes 48, and/or vectors 56 identified indata generated by the infrastructure sensor 22.

At a block 510 the server computer 36 receives the data requested at theblock 505 from the infrastructure sensor 22.

At a block 515 the server computer 36 transmits a first request to thevehicle 24. The first request specifies the vehicle 24 provide firstdata generated by the sensors 28 of the vehicle 24 while the vehicle 24is at a first location and orientation. The first request may specifydata specifying locations and orientations of objects 30, planes 48,and/or vectors 56 identified in the first data by the vehicle computer34.

At a block 520 the server computer 36 receives the first data requestedat the block 515 from the vehicle 24.

At a block 525 the server computer 36 identities the first location andorientation of the vehicle 24 on a map 26. The server computer 36 mayidentify the first location and orientation based on data received fromthe vehicle 24 specifying the first location and orientation.

At a block 530 the server computer 36 determines a first location andorientation of the infrastructure sensor 22 on the map 26 based on thedata from the infrastructure sensor 22 received at the block 510, thefirst data from the vehicle 24 at the block 520, and the first locationand orientation of the vehicle 24 identified at the block 525, e.g., asdescribed herein.

At a block 535 the server computer 36 transmits a second request to thevehicle 24. The second request specifies the vehicle 24 provide seconddata generated by the sensors 28 of the vehicle 24 while the vehicle 24is at a second location and orientation that is different than the firstlocation and orientation. The second request may specify data specifyinglocations and orientations of objects 30, planes 48, and/or vectors 56identified in the second data.

At a block 540 the server computer 36 receives the second data requestedat the block 535 from the vehicle 24.

At a block 545 the server computer 36 identities the second location andorientation of the vehicle 24 on the map 26. The server computer 36 mayidentify the second location and orientation based on data received fromthe vehicle 24 specifying the second location and orientation.

At a block 550 the server computer 36 determines a second location andorientation of the infrastructure sensor 22 on the map 26 based on thedata from the infrastructure sensor 22 received at the block 510, thesecond data from the vehicle 24 at the block 540, and the secondlocation and orientation of the vehicle 24 identified at the block 545,e.g., as described herein.

At a block 555 the server computer 36 determines an average location andorientation of the infrastructure sensor 22 on the map 26 based on thefirst location and orientation of the infrastructure sensor 22 on themap 26 determined at the block 530 and the second location andorientation of the infrastructure sensor 22 on the map 26 determined atthe block 550.

At a block 560 the server computer 36 stores data specifying the averagelocation and orientation of the infrastructure sensor 22 on the map 26determined at the block 555. Additionally or alternatively, the servercomputer 36 may transit data specifying the average location andorientation of the infrastructure sensor 22 on the map 26 to theinfrastructure sensor 22, and the infrastructure sensor 22 may storesuch data.

At a block 565 the server computer 36 transmits data specifying thelocation and orientation of the infrastructure sensor 22 on the map 26determined at the block 555 to the vehicle 24 or a second vehicle. Thevehicle computer 34 or a computer of the second vehicle operates therespective vehicle 23 or second vehicle in the autonomous and/orsemiautonomous mode based on the location and orientation of theinfrastructure sensor 22 on the map 26.

With regard to the process 500 described herein, it should be understoodthat, although the steps of such process 500 have been described asoccurring according to a certain ordered sequence, such process 500could be practiced with the described steps performed in an order otherthan the order described herein. It further should be understood thatcertain steps could be performed simultaneously, that other steps couldbe added, or that certain steps described herein could be omitted. Itshould be additionally understood that that other computers may performthe process 500. For example, the vehicle computer 34 may perform theprocess 500 as described for the server computer 36. In other words, thedescription of the process 500 herein is provided for the purpose ofillustrating certain embodiments and should in no way be construed so asto limit the disclosed subject matter.

Computing devices, such as the computer 34, 36, generally includecomputer-executable instructions, where the instructions may beexecutable by one or more computing devices such as those listed above.Computer-executable instructions may be compiled or interpreted fromcomputer programs created using a variety of programming languagesand/or technologies, including, without limitation, and either alone orin combination, Java™, C, C++, Visual Basic, Java Script, Perl, etc.Some of these applications may be compiled and executed on a virtualmachine, such as the Java Virtual Machine, the Dalvik virtual machine,or the like. In general, a processor (e.g., a microprocessor) receivesinstructions, e.g., from a memory, a computer-readable medium, etc., andexecutes these instructions, thereby performing one or more processes,including one or more of the processes described herein. Suchinstructions and other data may be stored and transmitted using avariety of computer-readable media.

A computer-readable medium (also referred to as a processor-readablemedium) includes any non-transitory (e.g., tangible) medium thatparticipates in providing data (e.g., instructions) that may be read bya computer (e.g., by a processor of a computer). Such a medium may takemany forms, including, but not limited to, non-volatile media andvolatile media. Non-volatile media may include, for example, optical ormagnetic disks and other persistent memory. Volatile media may include,for example, dynamic random-access memory (DRAM), which typicallyconstitutes a main memory. Such instructions may be transmitted by oneor more transmission media, including coaxial cables, copper wire andfiber optics, including the wires that comprise a system bus coupled toa processor of a computer. Common forms of computer-readable mediainclude, for example, a floppy disk, a flexible disk, hard disk,magnetic tape, any other magnetic medium, a CD-ROM, DVD, any otheroptical medium, punch cards, paper tape, any other physical medium withpatterns of holes, a RAM, a PROM, an EPROM, a FLASH-EEPROM, any othermemory chip or cartridge, or any other medium from which a computer canread.

In some examples, system elements may be implemented ascomputer-readable instructions (e.g., software) on one or more computingdevices (e.g., servers, personal computers, computing modules, etc.),stored on computer readable media associated therewith (e.g., disks,memories, etc.). A computer program product may comprise suchinstructions stored on computer readable media for carrying out thefunctions described herein.

The terms “in response to” and “upon” herein specify a causalrelationship in addition to a temporal relationship.

The disclosure has been described in an illustrative manner, and it isto be understood that the terminology which has been used is intended tobe in the nature of words of description rather than of limitation. Manymodifications and variations of the present disclosure are possible inlight of the above teachings, and the disclosure may be practicedotherwise than as specifically described.

What is claimed is:
 1. A system, comprising a computer including a processor and a memory storing instructions executable by the processor to: identify a location and an orientation of a vehicle on a map; and after identifying the location and the orientation of the vehicle on the map, determine a location of an infrastructure sensor on the map based on the location and the orientation of the vehicle, data from a vehicle sensor, and data from the infrastructure sensor including data from at least one of a radar sensor, a scanning laser range finder, a light detection and ranging device (LIDAR), an imaging device, or a camera of the infrastructure sensor.
 2. The system of claim 1, wherein the instructions further include instructions to identify the location and the orientation of the vehicle based on a location of an object on the map and a location of the object relative to the vehicle.
 3. The system of claim 1, wherein the instructions further include instructions to identify a first location of an object on the map and based on the data from the vehicle sensor, to identify a second location of the object relative to the infrastructure sensor and based on the data from the infrastructure sensor, and to determine the location of the infrastructure sensor based on the first and second locations of the object.
 4. The system of claim 1, wherein the instructions further include instructions to identify a first plane based on the data from the vehicle sensor, to identify a second plane based on the data from the infrastructure sensor, and to determine the location of the infrastructure sensor based on the first plane and the second plane.
 5. The system of claim 1, wherein the instructions further include instructions to identify a first vector based on the data from the vehicle sensor, to identify a second vector based on the infrastructure sensor, and to determine the location of the infrastructure sensor based on the first vector and the second vector.
 6. The system of claim 1, wherein the data from the vehicle sensor and the data from the infrastructure sensor includes point-cloud data, and the point cloud data from the infrastructure sensor data is used to determine the location of the infrastructure sensor.
 7. The system of claim 1, wherein the instructions further include instructions to identify a second location and a second orientation of the vehicle relative to the map, collect second data from the vehicle sensor while the vehicle is at the second location and in the second orientation, and to determine the location of the infrastructure sensor on the map based on the location and the orientation of the vehicle, the data from the vehicle sensor, the second location and the second orientation of the vehicle, the second data from the vehicle sensor, and the data from the infrastructure sensor.
 8. The system of claim 1, wherein the instructions further include instructions to determine an orientation of the infrastructure sensor on the map based on the location and the orientation of the vehicle, the data from the vehicle sensor, and the data from the infrastructure sensor.
 9. The system of claim 1, wherein the computer is remote from the vehicle and the infrastructure sensor, and wherein the instructions further include instructions to store the location of the infrastructure sensor on the map in the memory of the computer.
 10. The system of claim 1, wherein the instructions further include instructions to store the location of the infrastructure sensor on the map in a memory of the infrastructure sensor.
 11. The system of claim 1, wherein the instructions further include instructions to navigate a second vehicle based on the location of the infrastructure sensor.
 12. A method, comprising: identifying a location and an orientation of a vehicle on a map; and after identifying the location and the orientation of the vehicle on the map, determining a location of an infrastructure sensor on the map based on the location and the orientation of the vehicle, data from a vehicle sensor, and data from the infrastructure sensor including data from at least one of a radar sensor, a scanning laser range finder, a light detection and ranging device (LIDAR), an imaging device, or a camera of the infrastructure sensor.
 13. The method of claim 12, further comprising identifying the location and the orientation of the vehicle based on a location of an object on the map and a location of the object relative to the vehicle.
 14. The method of claim 12, further comprising identifying a first location of an object on the map and based on the data from the vehicle sensor, identifying a second location of the object relative to the infrastructure sensor and based on the data from the infrastructure sensor, and determining the location of the infrastructure sensor based on the first and second locations of the object.
 15. The method of claim 12, further comprising identifying a first plane based on the data from the vehicle sensor, identifying a second plane based on the data from the infrastructure sensor, and determining the location of the infrastructure sensor based on the first plane and the second plane.
 16. The method of claim 12, further comprising identifying a first vector based on the data from the vehicle sensor, identifying a second vector based on the data from the infrastructure sensor, and determining the location of the infrastructure sensor based on the first vector and the second vector.
 17. The method of claim 12, wherein the data from the vehicle sensor and the data from the infrastructure sensor includes point-cloud data, and the point cloud data from the infrastructure sensor is used to determine the location of the infrastructure sensor.
 18. The method of claim 12, further comprising identifying a second location and a second orientation of the vehicle on the map, collecting second data from the vehicle sensor while the vehicle is at the second location and in the second orientation, and determining the location of the infrastructure sensor on the map based on the location and the orientation of the vehicle, the data from the vehicle sensor, the second location and the second orientation of the vehicle, the second data from the vehicle sensor, and the data from the infrastructure sensor.
 19. The method of claim 12, further comprising determining an orientation of the infrastructure sensor relative to the map based on the location and the orientation of the vehicle, the data from the vehicle sensor, and the data from the infrastructure sensor.
 20. The method of claim 12, further comprising storing the location of the infrastructure sensor on the map in a memory of the infrastructure sensor. 